👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

bot:test
MARKERS="embedding"

Thanks!
Also cc @tjohnson31415 what I don't know how to do with kserve is correctly setup the ports so that you can just curl the predictor service. On my cluster I can seemingly hit the service if I set the port manually, but I don't think I should have to do that.

This connects:

curl http://granite-3-1-8b-instruct-predictor.a1-vllm-spyre:8000/v1/completions

But this refuses a connection:

curl http://granite-3-1-8b-instruct-predictor.a1-vllm-spyre/v1/completions

(I'm probably just missing something simple)

I think there was a way to fix the sign off commit suggestions directly from the PR page, but can't remember 🤔

Expected "Joe Runde [email protected]", but got "Joe Runde [email protected]".

yeah that confuses me, since my regular commits (that pass DCO) are all signed off with [email protected] :(

@prashantgupta24 for some reason github defaulted to my ibm email for the commit author, but my personal email for the signoff. 🤷

git commit --amend --reset-author
git push -f

fixes it